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ABSTRACT 

This paper is an approach to the construction of a European educational space (Novoa 
& Lawn, 2002), which is due to new modes of regulation in education. The policy under 
consideration is the institutional evaluation of schools carried out by the Portuguese General 
Inspectorate of Education. The aim is to explore how concepts and policies get 
“contaminated” by the European models (Barroso, 2003, 2006) and understanding how the 
regulation is outlined by the Inspectorates in some European countries, including Ireland, 
England, Wales, Scotland, France, the Netherlands and Belgium. This paper owes to the 
phenomena associated to “travelling policies” (Alexiadou & Jones, 2001), to “policy transfer” 
(Dolowitz et al, 2000; Stone, 2001), and to “policy borrowing” (Halpin & Troyna, 1995; 
Steiner-Khamsi, 2004). 

The authors’ perspective on the influences of the international movement of policies is 
free from simplistic and deterministic logics (Lingard & Rizvi, 2000), advocating that the 
internationalization of ideas come along with national reflections on how these ideas are 
materialized (Popkewitz, 1996). At the local/regional levels, the regulation of educational 
systems can be characterized as a growing ‘multi-regulation’ - that comes from a growing 
number of sources and a variety of tools (assessment, monitoring and sharing best practices) - 
which mingle with modes that exist in a more traditional, bureaucratic regulation (Afonso & 
Costa, 2010). Thus, each country has its own overview about the structures, and effects of 
globalization, which do not occur simultaneously, nor in the same way in the different 'nation 
states' (Lingard & Rizvi, 2000). 

Keywords: inspection, evaluation, regulation, Europeanization, education policy 


The role of European, Page 1 


Research in Higher Education Journal 


1. THE IMPORTANCE OF EVALUATION IN THE CONTEXT OF THE EUROPEAN 
UNION 

Since the last decades of the twentieth century assessment has become, globally, an 
institutional obligation in most developed countries. The Anglo-Saxon countries were the first 
to apply instruments for measuring the efficiency and quality, and were rapidly followed by 
large international organizations. Nowadays, the State is questioned about its traditional role, 
since the emergence of the Anglo-Saxon accountability, whose benchmarking and best 
practices permanently transform the concepts related to educational administration and 
increasingly influence management and governance. 

On the 1 st Conference on the Quality of Public Administrations in the EU, held during 
the Portuguese Presidency, in Lisbon, in 2000, a set of proposals were presented for the action 
program known as the "Lisbon Strategy". At that time, the European Council had the 
objective that the education and training system of the EU would become a global reference 
by 2010. The Lisbon Summit (2000) has characterized the Open Method of Coordination 
(OMC) as a means of disseminating best practices and of acquiring a greater convergence. 
Opposite to the classic process of European decision, the OMC consists in the establishment 
of guidelines for the EU Member States. It is based on a decentralized process of political 
construction among national governments, municipalities and civil society. It undertakes to 
set short-term objectives. The established policy is evaluated based on indicators and 
compared among the Member States. The process leads, in theory, to an emulation among 
Member States and it aims to develop good governance practices. 

The modernization and improvement in public services has been an issue expressed at 
different institutional levels and the object of recommendations emanated from the Parliament 
and the European Council. The objective is to modify and streamline public management 
systems, so that the quality of services is reflected socially in the form of a society oriented to 
a knowledge-based economy and, accordingly, demanding a larger commitment from 
educational systems and subsystems. At the summit in 2001 were issued recommendations 
that strengthen the fact that the “quality of school education must be assured at all levels an in 
all areas of education, regardless of any differences in educational objectives, methods and 
needs, and regardless of school excellence and rankings where they exist” (n ei 2 e 3). It’s 
emphasized the need: “to develop external evaluation in order to provide methodological 
support for school self-evaluation and to provide an outside view of the school encouraging a 
process of continuous improvement and taking care that this is not restricted to purely 
administrative checks” (l.f.) and “encouraging and supporting, where appropriate, the 
involvement of school stakeholders, including teachers, pupils, management, parents and 
experts, in the process of external and self-evaluation in schools in order to promote shared 
responsibility for the improvement of schools” (European Parliament and Council, 2001). 

These supranational procedures require forms of integration, incorporation and 
adaptation, focused “on the mediating actors who move between centre and locality” (Novoa 
& Lawn, 2002: 4), stressing the existence of several levels of regulation in what educational 
politics are concerned. Those levels are supranational, national and local and involve different 
institutions, actors and sources of regulation (Barroso, 2003, 2005, 2006, 2009). 

The governance of Europe can be “conceptualized as a multilevel system of 
governance” (Novoa & Lawn, 2002: 5), stimulating European cooperation and the exchange 
of transnational experiences to identify and disseminate effective methods of quality 
assessment (n. er 10). In most European countries, educational policies have focused on the 
problem of change in education, either global, institutional reforms, or according to a logic of 
promotion of innovation by identifying and disseminating best practices. With the 
phenomenon of globalization (and Europeanization), the production of global frameworks for 


The role of European, Page 2 


Research in Higher Education Journal 


interpreting the world tends to escape the national State, the borders and constraints of 
governments. As Barroso states (2003) it is 'transnational regulation', and the national 
regulation is a 'low intensity globalization'. The author (2003, 2005) refers to hybrid modes 
of institutional regulation and the crisis of the "State - educator", passed over by the State - 
market. This is clear in the proliferation of devices for evaluation and in the transfer of a 
control based on standards/rules for monitoring based on the results. The crisis of the State - 
Welfare is revealed and it is replaced by a State - evaluator, which expresses itself in 
promoting a competitive ethos through external evaluation (Afonso, 2001). 

2. REPERCUSSIONS IN THE PORTUGUESE CONTEXT 

The influence of the EU on the orientation and the legitimization of specific policy 
measures in the Member States allows to capture a set of modifications, transnational 
influences and interdependencies and, ultimately, contributes to the progressive structuring of 
a common European area of education. 

In Portugal, the pressure of the external evaluation is propagated indirectly and subtly, 
through this new soft way of governance of the EU. This is visible in the restoration of 
national exams for secondary school conclusion and the creation of national benchmarking 
tests in the last year of each cycle of basic education (4 th , 6 th and 9 th grades). However, since 
the academic year of 2004/2005, for the 9 th grade, aferition tests were replaced by national 
tests for Portuguese Language and Mathematics. This focus on the evaluation of school 
performance led to the creation of the Educational Evaluation Office (GAVE), a central 
bureau of the Ministry of Education with functions of planning, coordination, preparation and 
control instruments for external assessment of learning as well as supervision of the 
accurateness of the tests. Moreover, the pressure of the external evaluation is also expressed 
through ordered lists of schools, based on the results obtained by their students in national 
exams - the rankings; as well as international programs of assessment of student performance, 
such as the Program for International Student Achievement (PISA), conducted by the 
Organization for Economic Co-operation and Development (OCDE). The disclosure of the 
results into a logic of hierarchy of the participating countries feeds a media debate about 
"quality education", usually developed in a logic of “common sense” and not related to the 
education professionals’ “know how”, or even with expressed disdain for such “know how” 
(Afonso & Costa, 2007; Costa & Afonso, 2009, Afonso & Costa, 2009, and Carvalho, Afonso 
& Costa, 2009). 

3. THE POLICY OF EXTERNAL EVALUATION OF SCHOOLS IN PORTUGAL: 
THE NORMATIVE FRAMEWORK 

From the normative point of view, this new paradigm is reflected in the Law 31/2002, 
article 3 ld , which aims: “the improvement of the quality of the educational system, its 
organization and its levels of efficiency and effectiveness, to support the formulation and 
development of policies for education and training and ensure the availability of that 
management information system (paragraph a), article 3 ld , Decree-Law 31/2002). 

Through self-evaluation processes and external evaluation procedures, the external 
evaluation is seen as “a central instrument of policy education” (art. 3 ld ), and the relation 
among the micro, meso and macro levels of the educational systems is underlined. Article 9 th 
presents the first indicators for the evaluation of the parameters of schools to take into 
account. In Articles 14 th and 15 th general and specific objectives of the evaluation are set, as 
well as the dissemination of its results (Article 16 th ). 
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The logic that rules is that of for monitoring and steering. Its focus is on the 
production of relevant information about the quality of performances (Afonso, 1999). 
According to Ball (2004: 1116), this makes "the monitoring role of the state" easier, because 
it governs in a distant way - “governing without government [...] and the work with the 
educational institutions' knowledge (knowledge-work) becomes 'results',' performance levels', 
'quality forms'”. 

4. THE EXTERNAL ASSESSMENT OF SCHOOLS IN PORTUGAL: EUROPEAN 
INFLUENCES 

“Post-bureaucratic” instruments, such as the assessment of schools, increasingly focus 
on self-evaluation and autonomy for best practices to be adopted, while rule compliance 
instruments tend to lose their traditional prominence as key elements in policy making. This is 
built up through scientific knowledge, selected mainly from school effectiveness and school 
improvement literature, as well as from the inspectors’, principals’ and teachers’ professional 
know-how. There is the option for exterior knowledge, of an international nature (OECD, 
European Education Inspections, the Permanent International Conference of General and 
National Inspections of Education, etc). It is clear, in this process, the selection of certain 
kinds of products that insist on ‘quality’, ‘accountability’, ‘benchmarking’ and ‘best practices’ 
and its common reference to the new conceptions of the public administration (new public 
management), “educational modernization”. The references and sources of inspiration are 
clearly outlined with reference to models used in the Education Inspections of European 
countries. 

IGE’s participation in SICI, and the models used in European countries (e.g. Scotland 
and Northern Ireland), as well as the investigation coming from different countries, are quite 
relevant, as it will be examined in next section. 

4.1 CONTRIBUTIONS FROM EUROPEAN INSPECTIONS 

It is not strange in Europe the change of the modes of regulation of public policies 
regarding education. Due to social, political and economic modernization, in the last two 
decades, the States felt the need to create and/or strengthen inspection systems that would 
enable them to monitor the practices of the units that comprise them. Mainly from the nineties 
onward, there was an increase of the inspecting action in several European countries in what 
the public sector is concerned. 

The investigation that has been developed within the public policies at the level of the 
Inspectorate of Education stresses that the international debate on evaluation of schools has 
become central. Moreover, either due to managing reasons, or to increase autonomy and 
decentralization, many EU countries seek to implement a system of self-evaluation of schools. 
This has to do with the accountability, with the consequent restructuring of the external 
inspection (Meuret & Morlaix, 2003; Devos & Verhoeven, 2003; McNamara & O’Hara, 

2006; Perryman, 2006; Plowright, 2007; Wolf & Janssens, 2007). 

There seems to be a growing consensus: some forms of assessment can play a 
significant role in the efforts made to improve schools, as well as providing a reasonable level 
of public accountability, that prevails in democratic societies. Questions remain about how the 
balance can be achieved between these objectives. The framework provided by several 
authors illustrates convergent ideas, doubts and questions more or less common, based on the 
experience that the implementation of new policies has been offering. McNamara e O’Hara 
(2006) state that Ireland has been trying to develop an assessment of schools; this has been 
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done by balancing internal autonomy and accountability with external monitoring and 
inspection. The investigators refer to the progress that two evaluation projects provided, as 
well as their outcomes and consequences. The first, entitled Whole School Evaluation (WSE) 
was put into practice in the late nineties in 35 schools and it was completed in 1999. In 2003, 
comes the publication of a new framework for assessing schools, based on the WSE, and a 
second project called Looking at Our School (LAOS). The first evaluation of schools within 
this framework took place in 2004 and it highlights the emergence of an evaluative 
framework in a complex educational environment, in which teachers are highly resistant to 
what they consider to be interference to their professional autonomy. 

The evolution from the WSE to LAOS shows, however, a clear progression from an 
external monitoring to a self-evaluation (McNamara & O'Hara, 2006: 569). The school 
managers who participated in the projects stated that any form of external assessment was, by 
its nature, superficial and that it underestimated the achievements of schools in non-academic 
aspects. This raised many concerns among teachers who, by contrast, considered self- 
evaluation without external monitoring a great success (McNamara & O’Hara, 2006: 570). 
However, the authors point out drawbacks of this process: many aspects are not taken into 
account by the school, because the project does not provide guidelines regarding the criteria 
and research methods, and schools aren't used to collecting and analyzing data in order to 
gather evidence on which to base their opinions; the concept of self-evaluation wasn't 
assimilated yet and there is no development plan for the school based on the results. Among 
the schools surveyed, there is also a profound skepticism regarding the intervention by the 
Department of Education and Science (DES) in cases where the assessment has indicated 
problems. 

Thus, on the one hand, it would be useful that self-evaluation would be accompanied 
by more guidelines; on the other hand, it will take some time for a new structure to be 
assimilated. Finally, there are tools to be developed to improve the practices of schools whose 
self-assessment is not good. Plowright (2007) also wonders about the process of self- 
assessment of schools. The self-management of schools led to a greater pressure to take more 
responsibility for their development, progress, monitoring and review of educational 
provision. Aware of this fact in England, he questions the development of a school culture. 
One of his central questions is whether the school is developing an organizational culture of 
learning when it gets prepared for an external inspection through self-evaluation. Other 
questions that concern the author have to do with the inspection process, i.e., whether it 
contributes positively to school improvement, something also shared by Wolf and Janssens 
(2007). He also reflects on whether inspections can actually help schools to improve their 
ability to carry out self-evaluation; whether schools have the capacity to get involved in 
identifying their own problems honestly and rigorously. This last question meets Perryman 
(2006). Using the metaphor of the "panoptic gaze" to characterize the role of the Inspectorate 
in England and Wales, he launches a new question: the assessment the inspection makes is 
based on the self-assessment of schools, so, are the changes really assimilated by the school? 
(Perryman, 2007: 159). 

Ehren and Visscher (2006) focus on a theory of school improvement, through the 
Inspectorate, in the Netherlands. They describe two types of inspections according to two 
different scenarios. Schools with a weak innovation capacity and few external impulses 
should be helped by a more directive approach in which the inspector clearly points out the 
strengths and weaknesses of the school, the causes of this underperformance and potential 
ways of improvement, pressuring it in order to commit formally to develop an improvement 
plan. Instead, a school with high innovation capacity and strong external impulses should 
have a more reserved approach and inspectors would only stress their strengths and 
weaknesses. 


The role of European, Page 5 


Research in Higher Education Journal 


Wolf and Janssens (2007: 379) state that, in the Netherlands, new forms of 
accountability to improve school performance were introduced. These reforms were 
accompanied with discussions on the advantages and disadvantages of different mechanisms 
of external control in education, as well as an attempt to balance the different systems. 

According to Meuret and Morlaix (2003: 54), French schools are encouraged to 
develop an "evaluation culture". Although schools are given indicators that assist them in this 
process, these tools are used by only about 5% of schools. Teachers say this happens because 
they lack the time and expertise to develop any kind of evaluation. Self-assessment is not 
popular, although there is a greater receptivity to this process than the external assessment 
itself. Although the investigators consider that there isn’t evidence enough that self-evaluation 
can increase efficiency and improve the school, it seems to be useful. However, it is more 
praised by politicians than effectively used. 

Devos and Verhoeven (2003: 403) report that, in Belgium, policy makers feared that a 
greater autonomy would lead to a deterioration of the quality of schools, hence the need for 
the verification of compliance with the objectives outlined. Inspectors seek to convince the 
school that the quality is not only reached by an external monitoring, but also through self- 
evaluation, and try to promote a culture of self-evaluation that the authors compare to that of 
the OFSTED. 

The need to follow European trends is common to the Member States in general and to 
Portugal in particular. Indeed, the schools’ self-assessment has come to occupy the political 
agenda in different countries. There’s the belief that it may contribute to a significant 
improvement in schools’ provision, assuming more responsibility in the identification and 
resolution of their issues. Investigation indicates that this is an ongoing process and there are 
positive and negative indicators. Equally striking is some resistance of the schools, in 
implementing self-assessment, in spite of, in the cases of Ireland and France, useful tools for 
the schools to monitor in this process having been provided. This resistance appears to be 
associated with the reluctance to change itself, although self-assessment has been better 
received than the external assessment alone. For some investigators, it is clearer that self- 
evaluation can help to improve the performance of schools, but doubts remain about the role 
of the Inspectorate in this process. 

4.2. REGARDING THE MAIN SOURCES AND REFERENCES: THE SCOTTISH 
AND IRISH INFLUENCES 

In 2006, in Portugal, the 17 th Government launched a new experimental external 
assessment of schools Programme, coordinated by a School Evaluation Working Group, 
created under the Bureau of the Ministry of Education. The “School evaluation working 
group” (GTAE) was created within the Ministries of Finances and Public Administration and 
the Ministry of Education, for a year work. The main objective was to launch a national 
evaluation programme of non-higher education teaching establishments, with the aim to 
improve the quality of education and to create conditions to intensify schools autonomy. The 
work of this group, which was concluded in December 2006, conducted to the ministerial 
decision to proceed with the External Evaluation of Schools and to hand over the 
responsibility for its accomplishment to the IGE (Barroso et ah, 2007). 

Several different sources of information inspired the Portuguese policy of external 
evaluation. There are two models worth mentioning that are used in the UK. The Scottish 
model is a special case of transfer of explicit knowledge among the various member states, 
including Portugal: “because of the enormous popularity it enjoys in Europe and the educative 
dynamic that has promoted by giving schools the responsibility and the tools to question their 
own quality” (idem: 9). 
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According to GTAE, the handbook of self-assessment of the Scottish Inspection - 
“How good is our school” (2002, 2005) - is essential due to the clarity of the concepts and the 
formative nature of the model and the description of the evaluation criteria. “Exploring 
Excellence” is a more elaborate document and it introduces elements that were adapted to the 
Portuguese context, by making easier the internal benchmarking and by bringing a larger 
demand in the processes of comparability. 

Also stressed is the Northern Ireland evaluation model, whose self-evaluation model is 
noteworthy because it is part: “of a national strategy to improve education, which includes the 
evaluation of schools, assessment and improvement of the curriculum and educational 
services at the different educational levels, improvement of the quality of initial and 
continuous teacher training and an overhaul of the inspection methods so as to help guarantee 
quality.” Finally, there is the project ESSE (Effective School Self-Evaluation) created by 
SICI, which aimed to evaluate the consistency of self-assessment of schools and was also 
adopted in Portugal. 

4.2.1 THE IGE’S PARTICIPATION IN THE STANDING INTERNATIONAL 
CONFERENCE OF INSPECTORATES OF EDUCATION 

IGE’s participation in the activities of the Standing International Conference of 
Inspectorates of Education (SICI) is an example of institutional and international cooperation. 
This organisation was established in 1995 and brings together inspection services from a large 
number of European countries, and its activities are focused on the exchange of experiences, 
development of partnership projects and the organisation of scientific and professional events 
for the discussion of inspection methodologies and the continuous training of inspectors. The 
importance of this institutional relationship is reinforced by the General-Inspector in the 
introduction to the Activities Plan for 2007, emphasising the International Conference of 
Lisbon to be held in November, in the context of the Portuguese Presidency of the European 
Union, as well as the organisation of the workshop “Inspecting for Equality in Early 
Childhood Services” promoted by OFSTED. The aims of this activity were based on their 
importance in terms of knowledge circulation: to participate in the activities of SICI as a way 
of exchanging information, models and perspectives that may benefit the configuration and 
performance of IGE; to gather information that will contribute towards the improvement of 
the profile of the inspectors, in relation to the new challenges they face (Barroso et al., 2007). 
As a corollary of perspectives provided by SICI members, it must be stressed the following 
items that influenced the Portuguese context: the revision of the inspecting mission; the 
redefinition of the role of inspectors; the monitoring and methodological support for self- 
assessment of schools and the increasing of schools autonomy. 

SICI has contributed to the debate of education in Europe and the awareness that 
education is evolving rapidly, leading to new demands and expectations regarding the 
evaluation of its quality. There have been developments in educational systems, namely the 
trend to emphasize self-assessment of schools crossed with external evaluation, arising out of 
the propensity, at the European level, to increase school autonomy, leaving up to them the 
decision of their own modus operandi. There is also the need to take into account the 
individual characteristics and ambitions of the school organization and the learning that takes 
place outside the formal curriculum. Students, politicians and the public need assurance 
regarding the quality and the performance of schools. Technology is also changing the world 
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quite rapidly, including the ways people learn, communicate and share information. 
Therefore there should be an effort to modernize the tools of inspection, because the 
traditional fail to achieve the new goals. In its technical and scientific discourse, SICI also 
stresses that, in the European context, members of the EU set ambitious goals for their 
economies, societies and education systems to improve the quality and efficiency of education 
and facilitate access for all to lifelong learning, opening up education systems worldwide. 

In short, despite the differences that separate education systems in Europe, it seems 
there is a trend towards convergence in European education policies emerging in the last 
twenty years, resulting in a growing autonomy of schools, even if controlled by several 
methods such as assessment and monitoring practices; a trend towards decentralization of 
educational systems; the growth of the external evaluation at the level of education authorities 
and intermediate level schools through external evaluation and self-evaluation; and 
legitimization and promotion at different stages of a larger school choice by parents. These 
changes in modes of regulation seem to be due to political and economic factors, such as 
changing economic contexts that conduct the school system to a raise of the level of skills, to 
be more efficient and to adapt education to the needs of the labor market; the policy demands 
for spending on education are more effective and efficient, with a reduction of financial 
resources in some countries, cultural change in favor of a greater individualization of 
education; and finally, globalization and international comparison of school systems have a 
greater influence on national and local policies through the dissemination of 'governance 
models’, as well as management or educational models. 

CONCLUSION 

The transnational policies emanated from European institutions are 'itinerant policies', 
or ‘vernacular for globalisation’ (van Zanten, 2000). Local understanding of possible 
educational futures (Vongalis & Seddon, 2001: 1), influence policy and national and local 
educational practices, creating an environment of 'performativity' in school assessment 
(Lyotard, 1984, p.xxiv and Ball, 2004). Along with accountability, that assumes a central role 
rooted in the culture and practices of the public sector. Thus, there is a 'performance culture' 
related to the economic power, which is associated with the development of the comparisons 
of results, construction of standards and quality indicators. 

In terms of schools assessment, the change in the new modes of regulation of 
education is caused by progressive focusing on audit programs and external evaluation rather 
than on devices of a compulsory nature. Here, it must be stressed the importance of the 
integration of IGE in SICI, and the redefinition of its mission in the late '90s, both in the 
organization’s structure and its programs and projects. The institution that conducts external 
assessment in Portugal - IGE - has followed the evolution of new modes of regulation. 

Policies of external assessment also come from a recycling process of the inspection itself. 
Moreover, there was a key role played by a former General-Inspector, who had the knowledge 
about school assessment and came to influence indirectly the model of external assessment in 
Portugal. 

As stated, in the European education area there is a clear trend of an evolution of 'hard' 
modes of regulation for a 'soft', more persuasive regulation. In Portugal, the political- 
educational setting, in what the evaluation of non-higher education institutions is concerned, 
is based on the weakening of logical rules of inquisition and bets on the voluntary 
membership of actors, their involvement and collaborative work. Legal traditional penalties of 
the classical instruments (hard regulation) give rise to indirect modes of regulation, based on 
social sanctions and the socialization processes that are used as powerful mechanisms of 
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compliance. It is a transnational nature of regulation that contributes to the gradual framing of 

a European educational space, which is under construction and that is based on instruments 

that seek ideational power (Marcussen, 2004). 
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